Methods of and systems for measuring beacon stability of wireless access points

ABSTRACT

Methods of and systems for measuring beacon stability of wireless access points are provided. A method of determining a measure of likelihood that a designated wireless device exhibits at least one pathological characteristic includes determining a set of reference points associated with a designated wireless device, and retrieving attributes of the reference points, attributes associated with other wireless devices related to the designated wireless device, and/or statistical information. The statistical information includes a temporal distribution of detection of signals of reference points, a spatial distribution of the reference points, and/or a cardinality of the set or a subset of the reference points. The method further includes determining a measure of likelihood that the designated wireless device exhibits at least one pathological characteristic based on attributes of the reference points, attributes associated with the reception of signals, and/or the statistical information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. §119(e) of thefollowing application, the contents of which are incorporated byreference herein:

-   -   U.S. Provisional Application No. 61/353,936, entitled Methods of        and Systems for Measuring Beacon Stability of Wireless Access        Points, filed Jun. 11, 2010; and    -   U.S. Provisional Application No. 61/440,940, entitled Methods of        and Systems for Measuring Beacon Stability of Wireless Access        Points, filed Feb. 9, 2011.

This application is related to the following applications, filed on aneven date herewith, the contents of which are incorporated by referenceherein:

-   -   U.S. patent application Ser. No. TBD, Attorney Docket No.        2000319.176US1, entitled Systems for and Methods of Determining        Likelihood of Mobility of Reference Points in a Positioning        System;    -   U.S. patent application Ser. No. TBD, Attorney Docket No.        2000319.177US1, entitled Systems for and Methods of Determining        Likelihood of Relocation of Reference Points in a Positioning        System;    -   U.S. patent application Ser. No. TBD, Attorney Docket No.        2000319.178US1, entitled Systems for and Methods of Determining        Likelihood of Reference Point Identity Duplication in a        Positioning System; and    -   U.S. patent application Ser. No. TBD, Attorney Docket No.        2000319.179US1, entitled Systems for and Methods of Determining        Likelihood of Atypical Transmission Characteristics of Reference        Points in a Positioning System.

BACKGROUND OF THE INVENTION

1. Field of Invention

The invention generally relates to position estimates by a positioningsystem, and, more specifically, to the identification and/orquantification of features of positioning system reference points thatindicate the reference points could yield unreliable and/or deceptivepositioning information, and, in some cases, should not be relied uponfor positioning information.

2. Description of Related Art

In recent years, mobile and portable communication and computing deviceshave become ubiquitous, and wireless communication systems have expandedto meet the corresponding demand for connectivity. Mobile and portabledevices have no fixed locations and often accompany their users as theymove, and application developers have seized on the opportunity tocreate applications that update or adapt based on the locations of themobile devices running the applications. Intuitive examples include mapsthat update to indicate the current position of the device oradvertisements that adapt based on proximity to a particular point ofinterest.

In order to enable location-aware applications, device makers must maketheir devices capable of acquiring position information with minimaluser input. On the regulatory front, the FCC enhanced 911 rules mandatethat mobile telephones must be able to supply location information toemergency operators when making 911 calls. One conventional solution tothe problem of device positioning is GPS, which uses transmissions fromsatellites that follow carefully prescribed orbits. Unfortunately, GPSoften fails to cover indoor and densely developed urban areas, requiresdedicated hardware, and often suffers from slow time to first fix. Thus,more and more positioning systems are based on “beacons of opportunity”such as IEEE 802.11 access points and cellular base stations. They usetransmissions from existing wireless communication systems that arereceived by the standard hardware of the mobile device and combine themwith known information about the beacons to determine the position ofthe mobile device. Such systems effectively complement GPS coverageareas while providing fast time to first fix and requiring no dedicatedhardware.

There are currently numerous beacon-based positioning systems both inthe research community and in commercial and industrial deployments, andthey can be broadly divided into those that use pattern matching (alsoknown as fingerprinting) and those that use distance estimates to knownreference points. Pattern matching positioning technologies as developedby Placelab and others (“Practical Metropolitan-Scale Positioning forGSM Phones”, Chen et al.) estimate the position of the client device bymatching its observations to a map of received signal values collectedin the area.

In contrast, systems that use distance estimates explicitly estimatebeacon locations rather than simply mapping the patterns of receivedsignal strength from the beacons. Such systems then estimate the rangefrom the client device to the observed beacons based either on signalpropagation time or received signal strength (RSS).

Time-based systems use measurements of the time between transmission andreception of a signal to estimate the distance between the transmitterand the receiver. Such systems employ time of arrival (TOA) ortime-difference of arrival (TDOA) schemes to generate range estimatesfor use in a variety of algorithms to generate position estimates forthe user (U.S. RE38,808, Schuchman, et al; US 2002/007715 A1, Ruutu, etal). However, in asynchronous systems such as GSM and UMTS, additionalequipment is often installed at each cell at significant additional cost(U.S. Pat. No. 6,275,705 B1, Drane, et al; U.S. Pat. No. 6,526,039 B1,Dahlman, et al.; U.S. Pat. No. 6,901,264 B2, Myr).

Systems that use received signal strength (RSS) to estimate the distancefrom the mobile to the transmitting beacon use the fact that RSS isstrongly related to the distance from the transmitter to the receiver(“Indoor/Outdoor Location of Cellular Handsets Based on Received SignalStrength” by Zhu and Durgin). Well-known pathloss models show thatsignal power falls exponentially with distance, so knowledge of thepathloss exponent and other parameters such as antenna gain and transmitpower allows the positioning system to compute range estimates. Severalwell-known beacon-based positioning systems use this approach, notablyin the form of wi-fi positioning (WPS) based on IEEE 802.11 accesspoints.

BRIEF SUMMARY OF THE INVENTION Brief Summary of the Disclosure

In one aspect, the invention provides methods of and systems formeasuring beacon stability of wireless access points.

In another aspect of the invention, a method of determining a measure oflikelihood that a designated wireless device exhibits at least onepathological characteristic, the pathological characteristic causingerrors in position estimation to occur when the designated wirelessdevice is used as a landmark in a position estimation system if leftuncorrected includes determining a set of one or more reference pointsassociated with a designated wireless device. The one or more of thereference points being at least one of (i) a geographic position atwhich signals from the designated wireless device were detected and (ii)another wireless device from which signals were also detected by areceiver within a selected period of time during which the signals fromthe designated wireless device were detected by the receiver. The methodalso includes retrieving at least one of (i) attributes of the referencepoints of the set, (ii) attributes associated with the reception ofsignals from the designated wireless device and the another wirelessdevice within the selected period of time, and (iii) statisticalinformation about at least one of: a temporal distribution of times ofdetection of signals of reference points of the set, a spatialdistribution of the reference points, and a cardinality of the set or atleast one subset of the reference points. The method further includesdetermining a measure of likelihood that the designated wireless deviceexhibits at least one pathological characteristic based on at least oneof (i) attributes of the reference points, (ii) attributes associatedwith the reception of signals, and (iii) the statistical information.

In a further aspect of the invention, the measure of likelihood is abinary decision.

In yet another aspect of the invention, the measure of likelihood is arelative measure of a probability that the designated wireless deviceexhibits the at least one pathological characteristic.

In still a further aspect of the invention, the method also includesdetermining a measure of likelihood that a wireless device other thanthe designated wireless device exhibits at least one pathologicalcharacteristic based on the other wireless device having a relationshipwith the designated wireless device. The existence of the relationshipis based on the designated wireless device and the other wireless devicebeing within signal reception range of a same position within a givenperiod of time.

In another aspect of the invention, the designated wireless device has afirst identifier and the method also includes determining a secondmeasure of likelihood that a second designated wireless device exhibitsat least one pathological characteristic based on a comparison of thefirst identifier and a second identifier associated with the seconddesignated wireless device.

In a further aspect of the invention, the at least one pathologicalcharacteristic includes the designated wireless device being a mobiledevice which lacks a fixed geographic position.

In yet another aspect of the invention, the at least one pathologicalcharacteristic includes the designated wireless device exhibiting signaltransmission characteristics that are atypical of a wireless device of atype matching the designated wireless device.

In still a further aspect of the invention, the at least onepathological characteristic includes that the location informationassociated with the designated wireless device is not an accuratepresent location of the wireless device

In another aspect of the invention, the at least one pathologicalcharacteristic includes that an identifier associated with thedesignated wireless devices is also associated with at least one otherwireless device.

In a further aspect of the invention, the designated wireless device hasan identifier and the method also includes associating, in a referencedatabase, the identifier with an indication of the measure oflikelihood.

In yet another aspect of the invention, the designated wireless deviceis a WiFi-enabled access point.

In still a further aspect of the invention, the designated wirelessdevice is a mobile telephone transceiver installation.

In another aspect of the invention, for reference points of the set thatare another wireless device, the another wireless device of the set isat least one of a WiFi-enabled access point and a mobile telephonetransceiver installation.

In a further aspect of the invention, the attributes of the referencepoints of the set including location information specifying estimatedgeographic locations of the reference points of the set. The method alsoincludes grouping each reference point of the set into clusters of atleast one reference point based on the location information of thecorresponding reference point, and, for each cluster, determiningattributes of the cluster based on at least one of (i) attributes of thereference points of the set, (ii) attributes associated with thereception of signals from the designated wireless device and the anotherwireless device within the selected period of time. The method furtherincludes the determining the measure of likelihood being further basedon the attributes of at least one cluster.

In still another aspect of the invention, a system for determining ameasure of likelihood that a designated wireless device exhibits atleast one pathological characteristic, the pathological characteristiccausing errors in position estimation to occur when the designatedwireless device is used as a landmark in a position estimation system ifleft uncorrected includes a computer readable media includinginstructions that when executed by a computer system cause the computersystem to perform any of the methods recited above.

Any of the aspects recited herein can be used in combination with any ofthe embodiments disclosed herein.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 shows an example of a scan observation.

FIG. 2 shows an illustrative example of a graphic representation ofrelationship between beacons.

FIG. 3 illustrates the concept of a beacon moving from one location toanother.

FIG. 4 illustrates the concept of a beacon having Excess Range.

FIG. 5 shows a method for detecting a beacon relocation.

FIG. 6 shows a method for spatially clustering beacons.

FIG. 7 shows a further method of detecting a beacon relocation.

FIG. 8 shows yet another method of detecting a beacon relocation.

FIG. 9 shows still a further method of detecting a beacon relocation.

FIG. 10 shows another method of detecting a beacon relocation.

FIG. 11 shows yet another method of detecting a beacon relocation.

FIG. 12 shows a method of detecting a beacon relocation.

FIG. 13 illustrates a clustering technique for beacons.

FIG. 14 shows another method of detecting a beacon relocation.

FIG. 15 shows a method of building a beacon graph.

FIG. 16 shows a method of detecting a beacon exhibiting Mobility.

FIG. 17 shows another method of detecting a beacon exhibiting Mobility.

FIG. 18 shows a further method of detecting a beacon exhibitingMobility.

FIG. 19 shows yet another method of detecting a beacon exhibitingMobility.

FIG. 20 shows a method of detecting a beacon exhibiting Ubiquity.

FIG. 21 shows another method of detecting a beacon exhibiting Ubiquity.

FIG. 22 shows a further method of detecting a beacon exhibitingUbiquity.

FIG. 23 shows still another method of detecting a beacon exhibitingUbiquity.

FIG. 24 shows a method of detecting a beacon exhibiting Excess Range.

FIG. 25 illustrates a technique for determining the spatial extent of aset of scans.

FIG. 26 shows another method of detecting a beacon exhibiting ExcessRange.

FIG. 27 shows a further method of detecting a beacon exhibiting ExcessRange.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments relate to the use of wireless communication devices, such asIEEE 802.11 access points, as reference points for estimating thepositions of client devices such as cellular telephones, notebookcomputers, tablet computers, gaming systems, and other wireless-enableddevices. The word beacon is used in this document to refer to wirelesscommunication devices that are used as reference points (or landmarks)for positioning. Such a positioning system requires the creation andmaintenance of a database of beacon attributes. Beacons that move orotherwise defy accurate characterization (herein referred to aspathological beacons) can cause large errors when estimating theposition of client devices, so such beacons must be detected andcorrected in the database. Implementations of the invention providetechniques for detecting pathological beacons.

The term “reference point”, as used herein, refers to beacons as wellpoints and/or other devices that are used in the determination ofstability measures for the beacons. This is set forth in more detailbelow.

Under one aspect of the invention, distinct pathological beaconbehaviors, observables, and methods to quantify the likelihood that abeacon exhibits one or more of the various pathologies are defined. Asdescribed herein, these likelihoods are called “probabilities”, whichcan encompass a formal probability estimate as well as any otherexpression of relative likelihood. The individual pathologyprobabilities are extended to create an aggregate probability ofpathology that determines the overall stability and reliability of asingle beacon. Finally, the stability of individual beacons are extendedto quantify the reliability of an ensemble of beacons and propagatethrough a beacon graph.

In certain embodiments, stability is defined as a metric characterizingthe likelihood that a beacon will remain at the location (e.g. latitude,longitude and altitude) and continue to exhibit the attributes recordedin the beacon database and characteristics typical of the particularbeacon type.

Observables and Parameters

Certain embodiments characterize beacons either through directsubmissions or scans. Direct submissions involve the manual collectionof beacon identification and location information. FIG. 1 shows anexample of a scan observation. Scans 100 use signals 103 received fromthe beacon 102 to record where, when, and at what power level the beaconwas observed. Scans can further be divided into those collected byclient devices and those collected systematically by specializedscanning devices 101. Finally, by accumulating a set of scans, thebeacon database can be populated with attributes characterizing theexpected behavior of each beacon and the stability or likelihood thateach beacon will continue to conform to its expected behavior.

In some embodiments, the attributes that follow are accumulated overtime to show the history of the beacons and their relationships to oneanother. Such a history captures patterns of behavior that render abeacon particularly stable or unstable, and reveals features that wouldnot be evident in a single snapshot of the ensemble of attributes.

Submissions

Submissions, as used herein, describe the sharing of beacon informationthat has been collected or cataloged without using a scanning device.Submissions often come from users or institutions interested inimproving the performance of the wireless positioning system in theirhome, office, or campus, and they can be used to refine and augment scanrecords. Submissions may contain any or all of the following attributes:beacon identifier, beacon type, co-located beacons, beacon operator,beacon location, submission time, submission source, submission quality,and miscellaneous beacon hardware and software information.

Beacon identifiers include information such as MAC addresses of 802.11access points or the cell IDs of cellular base stations. In general,beacon identifiers can be used to distinguish between multiple beaconsof the same type. Beacon type includes information related to thestandards or protocols under which the beacon operates, e.g. IEEE 802.11(Wi-Fi) or 802.16 (WiMax), Bluetooth, GSM, etc. Co-located beacons arethe identifiers of other beacons that are either housed within the samehardware or installed in the same location as the beacon whoseinformation is being submitted.

The beacon operator is the individual or entity that owns or operatesthe beacon, not necessarily the same individual or entity that generatedthe submission. For instance, the operator could be a telecommunicationscompany that operates a network of Wi-Fi hotspots, and the submissionmight come from a bookstore that has one of the hotspots installed.

Beacon location information can take many forms including coordinatessuch as latitude, longitude, and altitude. Alternatively or in addition,beacon location can be described by metropolitan or regional identifierssuch as street address, building, floor, room, city, county, state, zipcode, region, or country. Finally, beacon location can be describedbased on physical characteristics of the land, water, and buildings inthe area.

Submission time and source describe when the submission occurred and theindividual or entity responsible for the submission. The submissionquality relates to the accuracy and trustworthiness of the submissionsource. For instance, a coffee shop might submit the locations of theirWi-Fi access points with a location accurate to within 10 meters whereasa university might submit locations accurate to 1 meter. In that case,the university submissions would be deemed to have a higher submissionquality.

Other possible submission information includes the manufacturer andmodel of the beacon, hardware and software configurations, installationdetails, and the beacon operator. Hardware configurations can includeinformation related to antennas, radios and amplifiers. Software(including firmware) configurations can include channel or frequencysettings, encryption, power control algorithms, and other modes orcapabilities. Installation details can include information related tothe position and orientation of the beacon as well as whether the beaconis indoor or outdoor, the orientation of directional antennas, andwhether the installation is temporary or permanent or fixed or mobile.For instance, a submission of a UMTS beacon could include the azimuth,antenna pattern, altitude, and downtilt of the sectorized antennas.

Scans

Referring again to FIG. 1, scans 100 record the characteristics ofsignals 103, 105 (e.g., received signal strength, time of transmission,time of reception) received from beacons 102, 104 along with anyavailable side information about the circumstances under which the datawas collected. In particular, while it is typically desirable to collectas much information as possible with each scan, scans may or may notinclude information about the location at which the scan was performed.Dedicated scanning devices 101 or client devices may perform scans. Forexample, an organization may deploy a dedicated scanning device 101 forthe express purpose of generating beacon records, whereas a useroperating a client device may collect scan data as a byproduct of normaluse (e.g., voice conversation, navigation, games, internetcommunication). While users may decide to purposely contribute scan datato the generation of a beacon database, the distinction lies in theprimary function of the scanning device rather than the intent of theindividual operating the scanner.

Scans may include any or all of the following attributes: beaconidentifiers, beacon type, scan location information, received signalcharacteristics, scan time and date, and scanning device information.Beacon identifiers can include both unique identifiers and non-uniqueidentifiers. Unique identifiers are often mandated by standards in orderto allow the coexistence of multiple beacons. For instance, MACaddresses in 802.11 and Cell Global Identification (CGI) in GSM are bothrequired to be globally unique identifiers by their respectivestandards. Non-unique identifiers can include user-configurable namessuch as SSID in 802.11.

Beacon type includes information related to the standards or protocolsunder which the beacon operates, e.g. IEEE 802.11 (Wi-Fi) or 802.16(WiMax), Bluetooth, GSM, UMTS etc.

Scan location information can include estimated scanner location,estimation accuracy, scanner location source, and other location relatedto the position of the scanner at the time of the scan. The estimatedscanner location can be expressed in coordinates such as latitude,longitude, and altitude, and it may also include velocity information.The estimation accuracy expresses the expected error in the estimate ofthe scan location. For instance, the accuracy of estimated positionprovided by a GPS system is often characterized as a function of thenumber, orientations, and signal strengths of the satellites used forpositioning.

The scanner location source contains information related to the systemsand methods used to estimate the location of the scanner during thecollection of the scan. For instance, scanner location may be providedby GPS or by the manual entry of coordinates by an individual operatingthe scanning device. In addition, the estimated scanner location may begenerated by the same positioning system that will in turn use the scanto update its beacon database. In any case, the scanner location sourcecan contain detailed records of the identities and signalcharacteristics of the reference points used to generate the positionestimate. Those records can then be used to describe the quality of thescanner location estimate or the quality of the reference points used toderive the location. For instance, if the scanner was positioned basedon Wi-Fi access points, the identities of those access points provideinformation about the quality of the estimated scanner location as wellas the quality of the estimated positions of the access pointsthemselves.

Other location information can include local identifiers and physicalcharacteristics. Local identifiers can include components of the civicaddress such as street address, building, floor, room, city, county,state, zip code, region, or country. Physical characteristics includeinformation related to the land, water, and buildings in the area aswell as information such as whether the scan was collected indoors oroutdoors.

Received signal characteristics include information related to thebeacon transmissions that were received by the scanning device. They caninclude received signal strength (RSS), channel or frequency of thesignal, and the noise level at the scanner.

Scanning device information can include scanner identifiers, scannertype, manufacturer and model information, and software and hardwareconfigurations. Scanner identifiers may or may not be globally unique.Scanner types may include both dedicated scanning devices, usedexpressly for the purpose of creating a database of beacons, and clientdevices, non-dedicated devices that collect scans intentionally or as abyproduct of other activities.

Beacon Attributes

By collecting relevant scans and submissions, certain embodimentsgenerate a series of attributes for each beacon in the database. Inaddition to being used to diagnose pathologies and assess overall beaconstability, beacon attributes serve important roles in positioning clientdevices. After pathology probabilities and stability values have beendetermined for a beacon, they can be stored as attributes and fed backinto the pathology and stability functions of the original beacon andother beacons in the database. In addition, as relevant informationaccumulates over time, beacon attributes may change, and the historicalrecord of attribute evolution can itself be an attribute of the beacon.

Beacon attributes may include one or more versions of any or all of thefollowing parameters: identifiers, position information, stabilityinformation, observation information, and beacon features. Beaconidentifiers can include both unique identifiers and non-uniqueidentifiers. Unique identifiers are often mandated by standards in orderto allow the coexistence of multiple beacons. For instance, MACaddresses in 802.11 and Cell Global Identification (CGI) in GSM are bothrequired to be globally unique identifiers by their respectivestandards. Non-unique identifiers can include user-configurable namessuch as SSID in 802.11.

Beacon position information can include estimated beacon location,estimation accuracy, and other information related to the scans orsubmissions used to estimate the position of the beacon. The estimatedbeacon location can be expressed in coordinates such as latitude,longitude, and altitude. The estimation accuracy expresses the expectederror in the estimate of the beacon location, which is often a functionof the number and attributes of scans and submissions used to estimatethe beacon position.

Other beacon location information can include local identifiers andphysical characteristics. Local identifiers can include components ofthe civic address such as street address, building, floor, room, city,county, state, zip code, region, or country. Physical characteristicsinclude information related to the land, water, and buildings in thearea as well as information such as whether the beacon is believed to beindoors or outdoors. Stability information contains information relatedto the likelihood that the beacon will exhibit any pathological behaviorand can include both aggregate stability values and values associatedwith individual pathologies.

Observation information relates to the times, places, and frequency withwhich the beacon was scanned and can include age, quiet time, duration,number of observations, observation types, observation frequency, andcoverage area. The age of a beacon is defined as the length of timesince the first observation of the beacon, and the quiet time is thelength of time since the last observation of the beacon. The duration isthe length of time between the first and last observations of thebeacon, number of observations is the total number of scans in which thebeacon has appeared. Observation types relate to the attributes of thescans or submissions relevant to the beacon.

Observation frequency describes the temporal distribution ofobservations. It can take the form of a simple average (totalobservations divided by total duration), a piecewise average(observations during a certain interval divided by the length of thatinterval), or more advanced statistics related to the precise temporaldistribution of observations.

Coverage area is the spatial equivalent of observation frequency in thatit describes, with some degree of granularity, the spatial distributionof observations. For instance, coverage area can be the radius of thesmallest circle that contains observations of the beacon, a set ofdisjoint rectangles of a certain size that contain the observations, ormore advanced statistics related to the precise spatial distribution ofthe observations.

Beacon features can include information such as the manufacturer andmodel of the beacon, hardware and software configurations, installationdetails, and the beacon operator—all of which can be derived from scansor submissions or both. Hardware configurations can include informationrelated to antennas, radios and amplifiers. Software (includingfirmware) configurations can include channel or frequency settings,encryption, power control algorithms, and other modes or capabilities.Installation details can include information related to the position andorientation of the beacon as well as whether the beacon is indoor oroutdoor, the orientation of directional antennas, and whether theinstallation is temporary or permanent or fixed or mobile. Finally,beacon features can include the identities of any other beacons that areeither housed in the same hardware or installed in the same location(co-located beacons).

Relationships

Under one aspect of the invention, when multiple beacons (e.g., beacons102,104 of FIG. 1) appear in the same scan (e.g., scan 100) orsubmission, a relationship is generated and attributes are assigned toeach relationship based on the circumstances under which the beaconswere observed. Specifically, a relationship captures the fact that twoor more beacons were within signaling range of a single point within awindow of time. As with beacons and their attributes, relationships canbe stored in a database for future use. Some embodiments indexrelationships using the unique identifiers of the related beacons, whichenables one to easily query the database for relationships including aparticular beacon.

In certain embodiments, a relationship between beacons that are believedto be outside of signaling range of one another indicates that one ormore beacons are exhibiting a pathology. Further, if the relationshipwas captured by a scan or submission without explicit locationinformation, then the relationship may indicate the presence of apathological beacon or beacons without directly delineating whichbeacons are pathological and which are not. For example, if two beaconsare believed to be positioned 100 km apart and have transmission rangesof 1 km, then an unpositioned scan in which signals were received fromboth beacons would indicate that one or both beacons were eitherincorrectly positioned or had an incorrectly characterized transmissionrange. The third alternative, which arises with some frequencyparticularly in end user devices, is that the beacons were not actuallyobserved within a given time period, but through caching or some similarbehavior the scanning device stored the scans as if they weresimultaneous and gave rise to a spurious relationship.

Relationships can have attributes of any or all of the following types:associated beacons, spatial features, temporal features, familyindicators, and quality metrics.

The list of associated beacons contains the identifiers of the beaconsconnected by the relationship, and spatial features describe the spatialrelationships between those beacons. Specifically, spatial features caninclude information related to the locations from which the relationshipwas observed and the estimated distances between the associatedbeacons—the physical distance spanned by the relationship. In addition,spatial features can include more detailed expressions or statisticalmeasures related to the area or areas in which the relationship has beenobserved.

Temporal features of a relationship can include the age, quiet time,duration, and observation frequency. The age of the relationship is thetime since the first observation of the relationship, and the quiet timeis the time since the last observation of the relationship. The durationis the length of time between the first and last observations of therelationship, and the observation frequency relates to the temporaldistribution of the observations of the relationship. Observationfrequency can be as simple as the average number of observations perunit time during the duration of the relationship or as sophisticated asstatistical measures of the burstiness of the observations and length ofgaps between observations.

Family indicators indicate whether the beacons connected by therelationship are believed to be a family based on scans, submissions, orside information. Specifically, a family of beacons is a group of two ormore beacons that would be expected to relocate as a group. Familiesoften arise when two or more beacons are owned and/or operated by asingle entity or individual. Thus, if the entity or individual relocatedto a new location, then the associated beacons would be moved to the newlocation as a group.

Quality metrics quantify the likelihood that the relationship isindicative of the current physical proximity of the set of associatedbeacons. For instance, a relationship that has not been observed for along time (large quiet time) might receive a lower quality value than anotherwise similar relationship that had been observed more recently. Ingeneral, quality metrics can be functions of any or all of theattributes of the associated beacons and the scans and submissionsrelated to those beacons.

As described in detail above, scans and relationships provideinformation about beacons. Specifically, one type of informationprovided by both scans and relationships is that a particular beacon ofinterest is within a certain distance of a given geographic location(the distance is related to the typical transmit distance of the beaconof interest). Another type of information provided by both scans andrelationships is the likelihood that the beacon of interest is stillwithin the distance at a given point in time. Thus, in the examples fordetecting pathologies that follow, scans and relationships can be usedinterchangeably. Therefore, although a particular example recites thatattributes of a relationship or attributes of a related beacon are usedwhen determining the probability that a beacon of interest is exhibitinga pathology, attributes of scans of the beacon of interest can be usedin place of the relationship or related beacon attributes.

Graphs

FIG. 2 shows an illustrative example of a graphic representation ofrelationship between beacons. As shown in FIG. 2, in certainembodiments, beacons become the nodes 201 and relationships become theedges 202 in a Beacon Graph 200. This is particularly useful inpropagating attributes between connected beacons, where two beacons areconsidered to be connected if there is a (possibly hop-limited) pathbetween the two beacons formed by traversing the edges of the graph. Forinstance, the stability of a particular beacon may be impacted by therespective stabilities of beacons to which it has relationships(referred to as neighbors) and the attributes of the relationships tothose neighbors, and each neighboring beacon's stability is in turnimpacted by the stabilities of its own neighbors, etc.

In addition, each graph edge 202 can be assigned a weight based on theattributes of the corresponding relationship and the connected nodes.The weight relates to the likelihood that the edge accurately representsthe current beacon geometry, and the weights can be used to generate aweighted distance between two nodes or an aggregate weight of edgescrossing a partition 205 of the graph. In other words, the weightsassigned to edges reflect the degree to which connected nodes affirm orcontradict one another's attributes, so a strong edge connecting twobeacons with contradictory attributes indicates that one or both beaconsare likely to have incorrect attributes. By extension, a path traversingmultiple strong edges can be used to find affirmation or contradictionbetween beacons even if they are not directly connected. For example, iftwo beacons whose attributes indicate that they are located manythousands of miles apart are connected by a few strong edges of a beacongraph, we can infer that one or both beacons may be incorrectlypositioned.

Edge weights can be functions of any relationship attributes orattributes of related beacons. For example, the weight of an edge couldbe determined by the number of separate observations of thecorresponding relationship as well as the time since the lastobservation of the relationship (Quiet Time) and the estimated distancebetween the connected beacons.

The Beacon Graphs defined under certain aspects of this inventionfacilitate the use of tools from the field of graph theory that will befamiliar to those having ordinary skill in the art. For instance,Dijkstra's Algorithm finds the minimum cost path between two nodes in aweighted, directed graph, and variations on the Ford-Fulkerson Algorithmcan be used to quantify the weights of graph edges crossing a partition.

Referring to FIG. 2, node 201 is a member of cluster 203, which isseparated from cluster 204 by partition 205. As addressed in some of theexamples that follow, the fact that node 201 is connected to nodes inboth clusters can be an indicator of pathology.

Note that the Beacon Graphs defined by some embodiments can have anunlimited number of directed or undirected edges between adjacentbeacons and that beacons need not be similar to be related and occupythe same graph. For instance, WiMax (IEEE 802.16), UMTS, and Wi-Fi (IEEE802.11) beacons could be related and placed in the same graph.

Beacon Pathologies

In simple terms, certain embodiments declare that a beacon is stable ifit can only be observed within a single coverage area, where theallowable size of a coverage area depends on the details of thestandards by which the beacon is governed. For the example of Wi-Fi(IEEE 802.11) access points, a reasonable coverage area would have aradius of several hundred meters.

There are several mechanisms through which a beacon can be renderedunstable, meaning that it can be or has been observed in multipledistinct coverage areas. Broadly, certain embodiments divide thepathologies into the following categories: relocation, mobility,ubiquity, and excess range. Each of the pathologies is defined below.

The scope of the invention includes taking particular actions uponidentifying a beacon as likely exhibiting one or more pathologies. Forexample, the beacon can be ignored by a positioning algorithm; thebeacon can be associated with an identifier of the pathology in adatabase, a report of beacons having likelihoods of exhibitingpathologies can be created, etc. Moreover, no specific action needs tobe taken at all, as merely determining likelihoods of beacons exhibitingone or more pathologies are within the scope of the invention.

Relocation

FIG. 3 illustrates the concept of a beacon moving from one location toanother. Certain embodiments declare that a beacon 301 exhibitsrelocation 300 when it is moved from one fixed location 302 to another304. If left uncorrected, relocation can cause the positioning system toposition users at the beacon's previous location 302 when they areactually at the beacon's new location 304. Relocation is often furtherobscured by the fact that multiple beacons may relocate together.

Relocations are further subdivided into two types: infrequent andfrequent. Infrequent relocations occur as often as several times peryear as individuals or organizations relocate permanently orsemi-permanently along with their beacons. For example, college studentsmoving to and from campus give rise to many infrequently relocatedbeacons. Frequent relocations occur as often as several times per month.For example, frequently relocated beacons might be moved with a touringperformance ensemble or an emergency response unit.

Mobility

Certain embodiments declare beacons to be mobile if they transmit whilebeing carried with a user or transported in a vehicle. Mobile beaconsoften appear in a large number of locations in quick succession, so theycan degrade user-positioning performance over large areas and for largenumbers of users.

Mobile beacons fall into the following categories: vehicular, personal,and asset.

Vehicular beacons are further subdivided into transit, professional, andindividual vehicular beacons. Transit beacons are permanently orsemi-permanently installed in buses, trains, boats, planes, or othervehicles that follow predictable routes.

Professional vehicular beacons are installed in commercial or industrialvehicles. For instance, beacons installed in limousines, taxis, deliverytrucks or other vehicles that follow unpredictable routes would beconsidered professional vehicular beacons. Individual vehicular beaconsare permanently or semi-permanently installed in automobiles, vans,trucks, or other personal vehicles that do not adhere to any predictableroutes.

Personal beacons are divided into portable and ad-hoc types. Portablebeacons are dedicated beacon devices that are carried by individuals,and, though they may be used in vehicles, are not permanently installedin any vehicle. Such devices are often referred to as Personal Hotspotsor Mobile Routers. Ad-hoc personal beacons are client devices such asnotebook computers or cellular telephones that are capable of acting asbeacons under certain configurations. One popular application of ad-hocbeacons is to “tether” a mobile telephone to another client device usingWi-Fi or Bluetooth in order to give the second device access to thecellular data network. In this example, the mobile telephone acts as thead-hoc beacon.

Asset beacons are beacons that are installed in or transported withgoods or other cargo. For instance, palletized cargo or shippingcontainers can contain beacons for use in inventory and tracking.

Ubiquity

Certain embodiments declare beacons to be ubiquitous if they haveidentifiers that are duplicated by other beacons and/or possess otherattributes that render them difficult or impossible to distinguish fromone another. That is, whereas non-ubiquitous beacons have globallyunique identifiers, ubiquitous beacons do not have any globally uniqueidentifiers. The result is that a ubiquitous beacon can appear to be inmultiple locations at the same time. Ubiquity is distinct fromrelocation and mobility because it does not require that a beacon bephysically moved from one location to another, and Ubiquity is exhibitedby an ensemble of beacons rather than a single pathological beacon.

Ubiquitous beacons fall into the following categories: factoryduplicated, reprogrammed, and dynamic. Factory duplicated beacons aredevices whose manufacturer has intentionally or inadvertently given thesame identifier to multiple devices. Reprogrammed beacons are devicesthat have been reprogrammed to intentionally or unintentionallyduplicate the identifiers of other devices. Dynamic beacons are deviceswhose identifiers change over long or short time scales andintentionally or unintentionally duplicate the identifiers of otherdevices. Dynamic ubiquitous beacons are particularly common in someenterprise Wi-Fi deployments in which identifiers are assigned by acentral controller and passed from beacon to beacon in order to give theappearance of seamless connectivity to the client device.

Excess Range

FIG. 4 illustrates the concept of a beacon having Excess Range. Certainembodiments declare a beacon 401 to have Excess Range if it isobservable over an area that significantly exceeds the coverage areas oftypical beacons of the same type. For example, a typical 802.11 beaconhas a coverage radius 402 on the order of 100 meters. An 802.11 beaconwith Excess Range could have a coverage radius 402 on the order of 1000meters. Like Ubiquity, Excess Range does not require that thepathological beacon be moved from one place to another, but unlikeUbiquity, Excess Range is a property of a single beacon and is basedentirely on the physical characteristics of the beacon and itsenvironment.

Certain embodiments declare a beacon to have Excess Range in response toa number of mechanisms including device configurations and environmentalfactors. Features such as nonstandard hardware 406 (e.g., high gaindirectional antennas) and firmware/software (e.g., selecting unusuallyhigh transmit power) often lead to Excess Range. Similarly, beaconsinstalled at high altitude 404 or near large open areas can often bedetected at ranges that exceed typical transmission distances.

Beacons exhibiting Excess Range fall into the following categories:environmental, configuration, and network.

Environmental Excess Range beacons include those installed or placed ona natural (e.g., mountain 404) or man-made structure (e.g., tower) thataffords an unobstructed path to distant locations by virtue of itsheight. Other mechanisms for environmental Excess Range include planareffects such as water bounce and waveguides. Water bounce 405 is aphenomenon in which signals from a beacon are strongly reflected by thesurface of a body of water. Waveguide effects arise from parallelreflecting surfaces such as “Urban Canyons” formed by tall buildings oneither side of a street. Waveguides have the effect of focusing signalenergy along the axis of the waveguide and extending the effective rangeof the signal.

Configuration-based Excess Range beacons are detected at unusually largedistances due to special hardware or software. Such configurations canbe as simple as setting the maximum transmit power used by the beacon toan unusually high level. Excess range can also arise from antennachoices. Directional antennas (e.g., parabolic dishes) achieve increasedrange in one or more directions by narrowing the pattern of the signalin azimuth and/or elevation. High-gain omnidirectional antennas achieveincreased range in a particular plane by narrowing the signal pattern inelevation.

Network-based Excess Range beacons use multiple transmitting devices toextend transmission range. Relay devices such as range extenders receivebeacon signals and then rebroadcast them, essentially re-amplifying thesignals so that they can be received at distances beyond the receptionrange of the original signals. Mesh networks consist of networks ofcooperating beacons in which transmissions may travel through the airbetween beacons over multiple hops and may give the appearance of ExcessRange for one or more beacons in the network.

Pathology Functions

Those having ordinary skill in the art will be familiar with the conceptof hard and soft decisions related to the detection of a certain binaryevent. In certain embodiments, a beacon is declared to either exhibit ornot exhibit a given pathology with either a hard or soft decision. Inembodiments utilizing a hard decision, the beacon is identified as beingdefinitely pathological or definitely not pathological. Embodimentsutilizing a soft decision assign a value (e.g., between 0 and 1, arelative score, etc.) to the probability that the beacon is actuallypathological. Any soft decision probability value can be converted to ahard decision by thresholding the pathology probability and roundingvalues above the threshold to the upper bound value (e.g., 1) and valuesless than or equal to the threshold to the lower bound value (e.g., 0).

Moreover, although the term “probability” often has a specificmathematical meaning as understood by one having ordinary skill in theart, the techniques set forth herein are also applicable to finding aprobability-like score, value, or measure of the likelihood of a beaconeither exhibiting or not exhibiting a given pathology. In addition,relative measures of pathologies can be determined using the techniquesdescribed here. Such relative measures represent the likelihood that abeacon is exhibiting a given pathology relative to another one or morebeacons. Thus, although the general description and specific techniquesdisclosed herein recite results in terms of a “probability”, is itunderstood that other values, scores, or measures of the likelihood of apathology is encompassed in this description.

Because the different pathologies can arise independently in the samebeacon, certain embodiments detect each pathology on its own. After theindividual pathology decisions have been made, any combination of thedecisions for the various pathologies can be used to create theaggregate stability value of the beacon.

The expression P(X|{A_(i)}) represents the probability that a beaconexhibits pathology X given that the beacon has the set of attributes{A_(i)}. P(X) is the a priori probability of pathology X over the entirepopulation of beacons, and P({A_(i)}) is the probability of theparticular n-tuple of attributes over a large collection of beacons. Theattribute set used to determine the probability of each pathology can beany subset up to and including the full set of attributes of the beaconitself and the attributes of any relationships and connected beacons. Inaddition, functions of beacon and relationship attributes can themselvesbe treated as attributes. Each individual probability expression andthreshold value used in the functions that follow can be derivedtheoretically or from empirical data using standard techniques that willbe familiar to those having ordinary skill in the art.

In the pathology probability expressions used in the examples thatfollow, the sets of attributes have been reduced for the sake ofclarity. In practice, it is often advantageous to reduce the set ofattributes either by ignoring certain attributes that are not relevantto the particular pathology or using functions that combine multipleattributes into a single parameter. However, in some implementations,additional attributes remain in the expressions.

General Hard Decision

Certain embodiments generate a hard decision by applying thresholds toone or more attributes and using binary logic to combine the differentattribute values into a single binary decision variable. Such harddecision functions are of the following form.

((A ₁ >b ₁)&(A ₂ >b ₂)& . . . &(A _(n) >b _(n)))|((A ₁ >c ₁)&(A ₂ >c ₂)&. . . &(A _(n) >c _(n)))| . . .

Alternatively, certain other embodiments use probability functions tocreate a likelihood ratio, and that ratio can be compared to a thresholdto produce a hard decision. Specifically the likelihood ratio testcompares the probability of two complementary hypotheses: the beacon ispathological, the beacon is not pathological (which is the same as 1minus the probability that the beacon is pathological). In the followinggeneral function, the ratio on the left exceeds threshold T whenpathology X is sufficiently probable, and the beacon is declaredpathological.

$\frac{P\left( X \middle| \left\{ A_{i} \right\} \right)}{1 - {P\left( X \middle| \left\{ A_{i} \right\} \right)}} > T$

General Soft Decision

Certain embodiments assign the probability that a beacon exhibits aparticular pathology using techniques tailored to the specificcharacteristics of each of the pathologies. In the methods that follow,the individual pathology probabilities take the following form based onthe well-known Bayes' Rule.

${P\left( X \middle| \left\{ A_{i} \right\} \right)} = \frac{{P\left( \left\{ A_{i} \right\} \middle| X \right)}{P(X)}}{P\left( \left\{ A_{i} \right\} \right)}$

Recall once again that the individual probability functions can bederived either from well-known theoretical results or empirical data bythose having ordinary skill in the art. For example, in certainimplementations, a particular set of beacons may be known or stronglysuspected of exhibiting a known pathology; the attributes of thesebeacons and/or the attributes of their relationships to other beaconscan then be correlated with the known or suspected pathology.

EXAMPLES

The following illustrative examples demonstrate how the pathologiesdefined by illustrative implementations of this invention can bedetected using the attributes of beacons, relationships, scans, andsubmissions. In general, the scope of the invention includes allpossible techniques used to make either soft or hard decisions on thepresence of the pathologies based on the set of attributes as definedherein. These examples are provided in order to better explain how suchtechniques might work. The examples should not be construed as anexhaustive set of all techniques covered by the invention, and they canbe combined and extended without limit to construct compound pathologydetection techniques.

The examples that follow recite various threshold values. Thresholds aretypically selected to meet performance goals for the detection system asquantified by two metrics well-known in the art: false alarm probabilityand miss probability. In this case, the false alarm probability is thelikelihood that the detector will mistakenly identify a beacon aspathological when in fact it is not. The miss probability is thelikelihood that a pathological beacon will not be identified aspathological. Intuitively, an effective pathology detector shouldminimize both the false alarm and miss probabilities. However, in anynon-trivial application, it will be impossible to achieve both a falsealarm and a miss probability of zero, and adjusting thresholds todecrease one probability will often increase the other. A standardpractice is to select the maximum allowable false alarm probability andthen adjust thresholds to minimize miss probability while stillremaining at or below the maximum false alarm probability. Overall, theselection of threshold values will be an application-dependent designdecision and can be performed through theoretical analysis or empiricaltesting.

Relocation Example 1

FIG. 5 shows a method for detecting a beacon relocation. In oneembodiment, relocation of a beacon B0 is detected by a method 500.Method 500 includes retrieving the attributes of beacons withrelationships to beacon B0 (neighboring beacons) and the attributes ofthose relationships (step 501). Next, a subset of neighboring beaconswhose stability value exceeds some threshold Ts (e.g., 0.99) isassembled (step 502) and whose relationship to beacon B0 has Quality(which, for example, can be based on the Quiet Time, estimated distancespanned, and number of observations of the relationship) exceeding somethreshold Tr (step 503).

Note that stable neighbors may come from different protocols. That is,some or all of an 802.11 beacon's stable neighbors may be UMTS cellularbeacons. The stability of the neighboring beacons can be determinedusing any combination of the techniques set forth herein or thecombination of techniques set forth herein and side information fromexternal sources (e.g. confirmation of municipal beacon locations from acity's public works department). Example stability functions are givenin a subsequent section of the document.

Next, the method determines the set of distances, {D}, from theestimated position of beacon B0 to the estimated positions of each ofits stable neighbors (where the estimated positions are stored in theattributes of each beacon) (step 504). The method then compares eachdistance in set {D} to a threshold, Td, and selects beacons whosedistance exceeds Td (step 505). The value of Td is determined based on atypical transmission range of the beacons. In other words, Td captures adistance beyond which two correctly positioned (i.e., having correctknown beacon locations) beacons should not be simultaneously observable.Next, beacons are selected whose relationship to beacon B0 has a QuietTime less than some threshold Tq (e.g. 7 days) (step 506). Finally, themethod declares beacon B0 to be relocated if any one or more of the samebeacons were selected in steps 505 and 506.

Example 2

Another implementation for detecting a beacon relocation is nowpresented. In this implementation, neighboring beacons are spatiallyclustered according to a method 600 shown in FIG. 6., while FIG. 7 showsa further method 700 of detecting a beacon relocation of beacon B0.First, attributes of beacons with relationships to beacon B0 areretrieved (step 501). Then, the neighboring beacons (which serve as thereference points in this example) are spatially clustered according tomethod 600 of FIG. 6.

According to method 600, estimated locations for each beacon related toB0 are retrieved (step 601), and an initial Cluster Center is designatedat the estimated location of beacon B0 (step 602). Then, for each of theneighboring beacons, Bi, the clustering method 600 determines thedistance to existing cluster centers (step 603). If there is a clustercenter within distance D1 (e.g., 1 km) of beacon Bi then beacon Bi isadded to that cluster (step 604). In certain implementations, the valueof distance D1 is related to and/or determined, in part, by the typicalcoverage area of beacons of the same type as beacon Bi. If no clustercenter is within distance D2 (e.g., 2 km) of beacon Bi, then a newcluster center is created at the location of beacon Bi (step 605). Incertain implementations, the value of distance D2 is determined, inpart, by the amount of distance that makes it likely, above a certainthreshold, that beacon B0 has relocated.

After creating the cluster centers, there should be no two clustercenters within distance D2 of one another, and no cluster should haveany beacons whose distance is greater than distance D1 from the clustercenter. Next, the clustering method addresses any beacons that have notyet been added to a cluster and adds them to the nearest cluster (step606).

After completing spatial clustering method 600, a stability value isassigned to each cluster (step 701) based on the stability of eachmember of the cluster and/or the number of beacons in the cluster. Forexample, the cluster is assigned to have the same stability as themaximum beacon stability in the cluster. Then the method determines thedistance from each cluster center to beacon B0 (step 702). Clusters withdistance greater than distance D3 (e.g., 2 km) from beacon B0 and havingstability greater than stability S1 are selected (step 703). In certainimplementations, the value of distance D3 is determined, in part, by theamount of distance that makes it likely, above a certain threshold, thatbeacon B0 has relocated. From the set of selected clusters, the methodselects clusters with a minimum Quiet Time less than time Tq (e.g., 30days) (step 704). If any clusters satisfied all three criteria: distancegreater than distance D3, stability greater than stability S1, and QuietTime less than time Tq, then the method declares beacon B0 to berelocated (step 705).

Example 3

FIG. 8 shows yet another method 800 of detecting a beacon relocation.First, attributes of beacons with relationships to beacon B0 areretrieved (step 501). Then the method spatially clusters the neighboringbeacons, using method 600, described above. After completing spatialclustering 600, an initial stability value, Si, is assigned to eachcluster Ci based on the number of neighbors in the cluster and the quiettime of the cluster (step 701). The number of neighbors in the clusteris related to the probability that the cluster has relocated as a unit,and the quiet time of the cluster is related to the probability that B0has relocated away from the cluster. The initial stability values can begenerated based on empirical observations or analytical expressionsrepresenting the probability that a cluster is providing validinformation about a related node as a function of cluster size and quiettime.

Next, the initial stability values, Si, is used to create jointstability values, Ji, for each cluster (step 801). The joint stabilityvalues quantify the fact that only one cluster can correctly reflectB0's current position, so the existence of multiple clusters createsconflicting information about B0's true location. For example, in oneimplementation, joint stability values are created by normalizing theinitial stability values such that the joint stability values sum to 1,see the following equation.

$J_{i} = \frac{S_{i}}{\sum S_{n}}$

In another implementation, the joint stability values are createdaccording to the following expression, which enforces the mutualexclusivity of clusters while remaining upper bounded such that jointstability Ji never exceeds initial stability Si.

$J_{i} = {S_{i}{\prod\limits_{k \neq i}\left( {1 - S_{k}} \right)}}$

After generating joint stability values, Ji, for the clusters, themethod determines which cluster, C0, corresponds to B0's estimatedlocation (step 802), and uses C0's joint stability value, J0, to assignB0's probability of relocation (step 803). In other words, J0 is closelyrelated to the probability that beacon B0 has been relocated. Thus, insome implementations, it may be sufficient to set the probability oflocation equal to J0 directly, though other applications may requireadditional scaling or offsets to return a satisfactory probability ofrelocation. Also note that the probability of relocation can bedetermined while B0's position is being estimated, and the positionestimate can be selected such that the probability of relocation isminimized. In other words, the system can determine multiple candidatepositions and then compare their respective relocation probabilities todecide which position is most likely to be correct.

Example 4

FIG. 9 shows still a further method 900 for detecting a beaconrelocation. In this implementation, positioned scans take the place ofthe neighboring beacons that were used as reference points in method800. The specific stability functions could take different forms whenusing scans instead of neighboring beacons, but the functional blocksare nearly identical, and, thus, the same reference numerals are used.First, the attributes of positioned scans in which beacon B0 wasobserved are retrieved (step 901). Next, the method spatially clustersthe positioned scans according to method 600, described above, andassigns an initial stability value to each cluster as described above inconnection with step 701.

Method 900 next uses the set of initial stability values to generate ajoint stability value for each cluster of scans 801. After generatingjoint stability values for the clusters, which cluster corresponds toB0's estimated location is determined (step 802). Finally, the jointstability value of the cluster of scans closest to the estimatedlocation of beacon B0 is used to determine the probability that B0 hasrelocated (step 803).

Example 5

FIG. 10 shows another method 1000 of detecting a beacon relocation. Themethod begins by retrieving the attributes of beacons with relationshipsto beacon B0 (step 501). Then the method selects any of the neighboringbeacons that belong to the same family as beacon B0 (located in the samehardware or owned by the same individual or entity and expected torelocate together) (step 1001). If any member of beacon B0's family hasa relocation probability above probability P1, then beacon B0 isdeclared to be relocated 1002.

Example 6

FIG. 11 shows yet another method 1100 of detecting a beacon relocation.The method begins by retrieving the attributes of beacons withrelationship to beacon B0 and the attributes of those relationships(step 501). Using a function of the Quiet Time of each relationship, aweight is applied such that relationships with small Quiet Times receivea larger weight than relationships with large Quiet Times (step 1101).Next, the method determines the distance from beacon B0's estimatedlocation to each of the estimated locations of beacon B0's neighbors(step 1102). Then, each distance is combined with its correspondingweight to create a weighted distance value for each relationship (step1103). From the set of weighted distance values, the Kth percentile(e.g., K=90) weighted distance is found (step 1104). Using techniquesfamiliar to those having ordinary skill in the art, the Kth percentilecan be found by arranging the weighted distance values in ascendingorder and selecting the value whose index equals ceil(0.9*L) where“ceil” represents the ceiling function—rounding up to the nextinteger—and L is the number of values in the list.

Finally, if the Kth percentile weighted distance exceeds some thresholdTd, then the method declares that beacon B0 is relocated (step 1105). Alarge Kth percentile weighted distance indicates that beacon B0 has alarge proportion of relationships that connect beacon B0 to beacons thatare positioned far away from beacon B0's current estimated position. Inturn, relationships to large numbers of apparently distant beaconssuggest that beacon B0 no longer resides at its estimated position.Combined with the fact that relationships with smaller Quiet Times (morerecently observed) receive higher weights, a large Kth percentileweighted distance means that not only was beacon B0 observed with asubstantial number of neighbors far from beacon B0's current estimatedposition, but the observations were relatively recent, and beacon B0 waslikely relocated.

Example 7

FIG. 12 shows a further method 1200 of detecting a beacon relocation.FIG. 13 illustrates clustering techniques used by method 1200 to detecta beacon relocation. The method begins by retrieving the attributes ofbeacon B0's relationships and the nodes to which beacon B0 is related(step 501). Next, using the estimated positions of beacon B0 and each ofits neighbors, the method calls for finding the distances between beaconB0 and each neighbor (step 1201). Using the set of distances, select theset of neighboring beacons whose distance from beacon B0 (1301 of FIG.13) is less than distance D (e.g. 2 km), and design the selected set asthe Home Cluster (step 1202 and shown as 1302 in FIG. 13).

Using the attributes of the beacons in the Home Cluster 1302, the methodfinds the total number of beacons, H, in the Home Cluster (step 1203)and the minimum Quiet Time, Q, out of Home Cluster members (step 1204).Finally, the method uses an equation of the following form to determinethe probability, P(R|H,T) that beacon B0 has been relocated (step 1205):

${P\left( {\left. R \middle| H \right.,T} \right)} = \frac{{P\left( {H,\left. T \middle| R \right.} \right)}{P(R)}}{P\left( {H,T} \right)}$

For clarity, other clusters 1303 and 1304 are shown in FIG. 13. Inaddition, distances between clusters 1305, 1306, and 1307, are shown.Although FIG. 13 shows the cluster distances as between the outerreaches of the clusters 1302, 1303, and 1304, in some implementations,other distances measures can be used, e.g., distances between clustercenters.

Example 8

FIG. 14 shows a further method 1400 of detecting a beacon relocation,which uses a method 1500 of building a beacon graph, as shown in FIG.15. Reference to FIG. 2 is also made to illustrate methods 1400 and1500. First, the method retrieves the attributes of beacons to whichbeacon B0 is related and the attributes of those relationships (step501). Second, a Beacon Graph of beacon B0 (201 of FIG. 2) and itsneighbors (e.g., 207) is constructed (step 1500 of FIG. 14, which ismethod 1500 of FIG. 15) by retrieving relationships between the neighborbeacons (step 1501). For beacon B0 and each neighbor, the method createsa node in the Beacon Graph (step 1502), and creates an edge for eachrelationship connecting a pair of beacons (step 1503).

Next, the set of spatial distances {Dij} between each pair of connectednodes Ni and Nj is found by using the estimated positions of the beaconsBi and Bj corresponding to those nodes (step 1401). Then the methodpartitions the Beacon Graph 200 into two sub-graphs 203, 204 such thatthe partition 205 crosses only edges 206 for which the nodes haveestimated positions separated by at least distance D1 (e.g. 2 km) (step1402). In certain implementations, the value of distance D1 isdetermined, in part, by the amount of distance that makes it likely,above a certain threshold, that beacon B0 has relocated. The minimumweight partition can be found using an exhaustive search or by usinggraph partitioning algorithms found in the literature of graph theory.

If such a partition exists, then the method next finds the node(s) 201connected to edges 206 crossing the partition 205 (step 1403), anddivides them into two subsets based on the sub-graph 203, 204 to whichthey belong (step 1404). The distances spanned by the edges crossing thepartition are much larger than the transmission range of the beacons, sothe partition indicates that nodes in one of the subsets have beenrelocated.

Finally, if beacon B0 is in the smaller of the two subsets, the methoddeclares that beacon B0 and the other beacons whose nodes are in thatsubset to be relocated, and indicates that the beacons (if there is morethan one member of the smaller subset) form a family (step 1405). Assuch, beacon B0 and its family members can be expected to move togetherin future relocation events.

Mobility

As described above, certain embodiments declare beacons to be mobile ifthey transmit while being carried with a user or transported in avehicle. In other words, the beacons are determined to exhibit thepathology of Mobility.

Example 1

FIG. 16 shows a method 1600 of detecting beacon Mobility. First, themethod retrieves attributes of beacon B0 (step 1601). Next, beacon B0'sidentifiers and manufacturer information is compared to a “black list”of manufacturers and identifiers known to correspond to mobile devices(step 1602). If beacon B0's identifiers and/or manufacturer informationappear in the black list, then the method declares that beacon B0 ismobile (step 1603).

It is common practice among manufacturers to designate specific patternsof unique identifiers (e.g., MAC address ranges for 802.11 devices) formobile and portable devices. Further, there are some manufacturers thatexclusively produce mobile and portable devices, so manufacturerinformation alone may suffice to indicate that a given beacon is mobile.

Example 2

FIG. 17 shows another method 1700 of detecting beacon Mobility. First,the method finds the number of relationships containing beacon B0 (step1701). Second, for each of beacon B0's neighbors, the number ofrelationships is found (step 1702). Then, using techniques familiar tothose with ordinary skill in the art, the Kth percentile (e.g., K=99)number of relationships over the set of beacon B0's neighbors is found(step 1703). In certain implementations, the value of K is determined,in part, by the expected number of mobile beacons as a fraction of theentire beacon population and the difference between the expected numberof relationships for mobile and non-mobile beacons. Finally, if beaconB0's number of relationships exceeds the Kth percentile number ofrelationships among beacon B0's neighbors, then the method declaresbeacon B0 to be mobile (step 1704).

In general, mobile beacons have the opportunity to be observed withneighbors from a spatial area whose radius is significantly larger thana typical beacon transmission range. Thus, mobile beacons often formextremely large numbers of relationships that far exceed the numbers ofrelationships accumulated by their non-mobile neighbors.

Example 3

FIG. 18 shows a further method of detecting beacon Mobility. First, themethod maintains a record of every instance in which beacon B0 has beendeclared relocated and repositioned in the database (step 1801). Then,from those instances of relocation and repositioning, the number ofrelocation that have occurred within the past T1 time intervals (e.g.,365 days) is determined (step 1802). Finally, if the number ofrelocation and repositioning events exceeds threshold R (e.g., 5), thenthe method declares beacon B0 to be mobile (step 1803).

In certain implementations, the values of number T1 and threshold R arejointly determined, in part, by the frequency of beacon attributeupdates in the system and the distinction between a frequently relocatedbeacon and a truly mobile beacon. A frequently relocated beacon willreside in a fixed location sufficiently long that the system will beable to detect the relocation, reposition the beacon and use it forreliable user positioning. However, a mobile beacon will continue tochange its location faster than the system can react, and it will neverprovide reliable user positioning.

Example 4

FIG. 19 shows yet another method 1900 of detecting beacon Mobility.First, the method finds the total number, N, of relationships containingbeacon B0 (step 1901). Next, the method determines how many of beaconB0's relationships have duration (stored in the attributes of eachrelationship or determined based on the time difference between thefirst and last observations of the relationship) less than time T (e.g.,1 day) (step 1902). In certain implementations, the value of time T isdetermined, in part, by the distributions of relationship durationstypical of mobile and non-mobile beacons. Finally, find the probabilitythat beacon B0 is mobile is determined (step 1903) by using a functionof the following form:

${P\left( {\left. M \middle| N \right.,Z} \right)} = \frac{{P\left( {N,\left. Z \middle| M \right.} \right)}{P(M)}}{P\left( {N,Z} \right)}$

Often, mobile beacons will have both a large number of totalrelationships and a large fraction of short duration relationships. Inparticular, mobile beacons that travel in an apparently random fashion(as opposed to a repeated route) will primarily accumulate shortduration relationships due to the fact that they may only be in physicalproximity to a large subset of their neighbors on only one occasionduring the life of the beacon.

Ubiquity

As described above, certain embodiments declare beacons to be ubiquitousif they have identifiers that are duplicated by other beacons and/orpossess other attributes that render them difficult or impossible todistinguish from one another. In other words, the beacons are determinedto exhibit the pathology of Ubiquity.

Example 1

FIG. 20 shows a method 2000 of detecting a beacon exhibiting Ubiquity.First, the method retrieves the attributes of beacon B0's relationshipsand neighboring beacons (step 501). Then neighboring beacons whoseStability values are above some threshold S1 (as stored in the databaseof beacon attributes) and whose relationships to beacon B0 have Qualityvalues above some threshold Q1 are selected (step 2001). In certainimplementations, the values of threshold S1 and threshold Q1 will bedetermined, in part, by typical values of Stability and Quality in thesystem and various performance considerations.

Next, the method spatially clusters the set of selected neighbors usingmethod 600, described above, and defines an observation interval foreach cluster (step 2002). To define an observation interval, oneimplementation of the method 2000 sets the beginning of the interval asthe time of the first observation of any relationship between beacon B0and a member of the cluster, and, for the same cluster, sets the end ofthe interval as the time of the last observation of any relationshipbetween beacon B0 and a member of the cluster. Overall, the times ofevery observation of beacon B0 with any member of the cluster shouldfall within the cluster's observation interval.

Finally, if the intervals for any two clusters overlap (the intersectionof the intervals forms an interval of nonzero duration), then the methoddeclares beacon B0 to be ubiquitous (step 2003). Given the fact thatspatial clusters are separated by a distance greater than beacon B0'scoverage radius, overlapping observation intervals indicate that beaconB0 was observed in multiple distinct locations during a single intervaland therefore must be ubiquitous.

Example 2

FIG. 21 shows another method 2100 of detecting a beacon exhibitingUbiquity. First, the method begins by retrieving scans in which beaconB0 was observed and the scan attributes contain reliable scan locations(step 901). The reliability of a scan location is related to the sourceof location associated with the scan. For example, with a relativelyclear view of the sky, a GPS-generated location will likely be morereliable than a location estimate from a Wi-Fi based positioning system,which, in turn, is likely more reliable than a cell tower-based positionestimate. In general, each positioning technique can be associated witha generally accepted positioning error, which can be used to rank thetechniques against each other in terms of reliability, with lowerpositioning errors having higher reliability measures. The scans couldbe from any device or devices that supplied reliable records of theirlocations (e.g., GPS). Next, scan times in the scan attribute data aresorted in ascending chronological order (oldest to newest) by the timeat which the scans were collected, and the amount of time that passedbetween each scan and the next is found (step 2101). In this step, it isacceptable to have zero time difference between subsequent scans, thoughit may prove useful to replace zero time difference with a smallconstant in some implementations to prevent a divide by zero condition.Next, the method spatially clusters the selected scans according tomethod 600 described above.

Now that each scan belongs to a cluster, the method assigns each scan tothe location of its cluster center such that scans from the same clusterare associated with the same location, and then finds the distancebetween each scan and the following scan in the ordered list of scanscreated in step 2101 (step 2102). Next, the velocity (distance dividedby time) necessary to move from one scan to the next scan in the orderedlist is determined (step 2103). Finally, if any pair of scans wouldrequire a velocity greater than V1 (e.g., 1600 km per hour), then themethod declares beacon B0 to be ubiquitous (step 2104).

In certain implementations, the value of V1 is determined, in part, bythe typical speeds at which a mobile beacon might travel from onelocation to another. Hence, if the velocity necessary to travel from onecluster of scans to another exceeds the reasonable speed of a mobilebeacon, then beacon B0 exhibits Ubiquity by appearing in multiplelocations effectively simultaneously.

Example 3

FIG. 22 shows a further method 2200 of detecting a beacon exhibitingUbiquity. First, the method retrieves the attributes of beacon B0 (step2201). Then beacon B0's identifiers and manufacturer information iscompared to a “black list” of known ubiquitous beacons and beaconmanufacturers (step 2202). If beacon B0's identifiers and ormanufacturer information appear on the black list, then the methoddeclares beacon B0 to be ubiquitous (step 2203).

In many cases, beacon identifiers are assigned or governed by astandards body (e.g. IEEE assigns MAC address ranges for 802.11beacons). Beacon identifiers that give evidence of noncompliance withstandards have often been reconfigured without appropriate safeguardsagainst duplication and can cause instances of Ubiquity. Further,certain manufacturers assign non-unique identifiers either randomly orsystematically to their beacons (often in defiance of standards) and canbe recognized through the analysis of market research and/or empiricaldata and added to a black list of ubiquitous beacon manufacturers.

Example 4

FIG. 23 shows yet another method 2300 of detecting a beacon exhibitingUbiquity. First, the method retrieves the attributes of beacon B0, itsneighbors, and the relationships between beacon B0 and each neighbor(step 501). The method spatially clusters selected neighbors accordingto method 600 (and as shown in FIG. 13), described above, and finds theQuiet Time for each cluster by taking the minimum of the Quiet Timevalues for the relationships between beacon B0 and members of thecluster (step 2301).

Next, the number of clusters, C, which include more than threshold K(e.g., 5) beacons and whose Quiet Time is less than threshold Q (e.g., 7days) are determined (step 2302). In certain implementations, the valuesof thresholds K and Q are determined, in part, by the empirical ortheoretical probability distributions of cluster size and Quiet Timesuch that the number of clusters with size greater than threshold K andQuiet Time less than threshold Q provides a meaningful differentiatorbetween ubiquitous and non-ubiquitous beacons. A sufficiently largethreshold K reduces the probability that a given cluster will consist ofa single family of beacons relocated as a group to beacon B0's location.A sufficiently small threshold Q insures that beacon B0 is in factubiquitous rather than mobile or relocated. However, an overly largethreshold K or small threshold Q will cause the system to ignoreinformative clusters that could provide evidence of Ubiquity, so theselections of thresholds K and Q must be informed by empirical and/ortheoretical understanding of beacon behavior and determined according tothe performance requirements of the system.

Next, the method retrieves the identity of beacon B0's manufacturer, F(step 2303), and uses an equation of the following form to determine theprobability, P(U|C,F), that beacon B0 is ubiquitous (step 2304):

${P\left( {\left. U \middle| C \right.,F} \right)} = \frac{{P\left( {C,\left. F \middle| U \right.} \right)}{P(U)}}{P\left( {C,F} \right)}$

Even if a particular manufacturer does not produce exclusivelyubiquitous or non-ubiquitous beacons, certain manufacturers may producebeacons with a greater or lesser probability of Ubiquity and change theprobability of Ubiquity as a function of C, the number of selectedclusters.

Excess Range

As described above, certain embodiments declare beacons to have ExcessRange if it is observable over an area that significantly exceeds thecoverage areas of typical beacons of the same type. In other words, thebeacons are determined to exhibit the pathology of Excess Range.

Example 1

FIG. 24 shows a method 2400 of detecting a beacon exhibiting ExcessRange. FIG. 25 illustrates a technique for determining the spatialextent of a set of scans, which is useful in method 2400.

First, the method retrieves the attributes of scans in which beacon B0was observed (step 901). Then those scans whose attributes containlocation information from a reliable source (e.g., GPS) are selected(step 2401). Next, the spatial extent, E, of the selected scans aredetermined (step 2402) by finding the length of a diagonal (line 2504 ofFIG. 25) of a smallest rectangle 2505 that contains selected scans 2502of beacon B0 2501. Finally, if extent E exceeds some threshold E1, thenthe method declares beacon B0 to have Excess Range. In certainimplementations, the value of threshold E1 is determined, in part, bythe expected transmission range of a beacon of beacon B0's type. Whilethe above example uses a rectangle to determine spatial extent, this ismerely illustrative and other geometric figures are within the scope ofthe invention, e.g., circles, triangles, and/or other polygons.

Example 2

FIG. 26 shows a method 2600 of detecting a beacon exhibiting ExcessRange. First, the method retrieves the attributes of beacon B0, itsneighbors, and the relationships between beacon B0 and its neighbors(step 501). Then, an Estimated Distance attribute for each relationshipis extracted from the retrieved information or determined (step 2601).

In certain implementations, the Estimated Distance of a relationship isdetermined based on the locations and received signal strength values ofthe scans used to define the relationship. For instance, using signalpathloss models familiar to those having ordinary skill in the art, thesystem can estimate the distance from the scan location to the locationsof beacons from which signals were received. Basic principles of planargeometry state that the shortest path between two points forms astraight line, so the scan with the minimum sum distance to a pair ofbeacons will be the scan closest to the straight line connecting the twobeacons. Thus, subject to certain assumptions about the isotropy of scanlocations and the number of scans, the scan with the minimum sumdistance to a pair of beacons should lie on or about the line betweenthose beacons, and the sum distance to the scan location should form agood estimator of the distance between the beacons—the EstimatedDistance of the relationship.

Next, the method ignores relationships with neighboring beacons whichthemselves have a probability of Excess Range greater than somethreshold E1 (step 2602). In certain implementations, the value ofthreshold E1 is determined, in part, by performance considerationsrelated to the sensitivity of the user-positioning algorithm to beaconshaving Excess Range and the degree to which a beacon with Excess Rangecan distort the observed attributes of its neighbors. For example, ifbeacon B0 has a neighbor exhibiting Excess Range, then that neighbor cancause the mistaken impression that beacon B0 itself has Excess Rangebecause the two beacons were seen together even though their estimatedpositions indicate that their separation should preclude thesimultaneous observation of beacons with typical transmission range.

Finally, if any Estimated Distance of a relationship between beacon B0and a selected neighbor exceeds threshold D1, then the method declaresbeacon B0 to have Excess Range (step 2603). In certain implementations,threshold D1 is determined, in part by the typical transmission rangesof beacon B0 and its neighbors.

Example 3

FIG. 27 shows a method 2700 of detecting a beacon exhibiting ExcessRange. Reference is also made to FIG. 25 for illustrative purposes.First, the method retrieves the attributes of scans in which beacon B0was observed and reliable location information is available (step 901).Second the method finds the distance (2503 from FIG. 25) from beaconB0's estimated position 2501 to the position of each selected scan 2502(step 2702). Third, using techniques familiar to one with ordinary skillin the art, the Gth percentile (e.g., G=90) distance, Ds, from beaconB0's estimated position to the positions of the selected scans is found(step 2703). In certain implementations, the value of percentile G isdetermined, in part, by the quality of the location informationavailable in the attributes of the scans. If it is likely that a certainpercentage of the scans contain spurious location information, thendecreasing percentile G allows a larger fraction of scans to bedisregarded.

Next, the method retrieves the attributes of beacon B0's relationships(step 2704), and extracts the Estimated Distance, as described above, ofeach relationship (step 2705). Then the Ith percentile EstimatedDistance, De, of the set of relationships is found (step 2706). Incertain implementations, the value of percentile I is determined, inpart, by the quality of the Estimated Distance values and the likelihoodthat some fraction of relationships would contain inaccurate values.Decreasing percentile I allows the system to disregard an increasingfraction of relationships having large Estimated Distance values.

Next, the attributes of beacon B0's neighboring beacons for whichposition information is available is retrieved (step 2707), and thedistance between beacon B0's estimated position and the position of eachneighbor is determined (step 2708). Then the method determines the Hthpercentile distance, Dn, between beacon B0 and its neighbors (step2709). In certain implementations, the value of percentile H isdetermined, in part, by the probability that some fraction of beaconB0's neighbors have incorrect positions.

Finally, the method uses an equation of the following form to determinethe probability, P(L|Ds,Dn,De), that beacon B0 is ubiquitous:

${P\left( {\left. L \middle| D_{s} \right.,D_{n},D_{e}} \right)} = \frac{{P\left( {D_{s},D_{n},\left. D_{e} \middle| L \right.} \right)}{P(L)}}{P\left( {D_{s},D_{n},D_{e}} \right)}$

Stability Functions

Certain embodiments aggregate individual pathology probabilities to forman overall stability metric. In general the aggregation function takesthe following form:

S=f({P _(j)})

In the equation above, S is the stability value, and function f takes inthe set of pathology probabilities, where Pj is the probability ofpathology j.

Stability Examples

The following functions are illustrative examples of how to aggregatethe various pathology probabilities into a single stability value, S.

In the first example, stability ranges between 0 and 1 and is negativelyrelated to the probability of the most probable pathology (see equationbelow). Thus, larger S would indicate greater stability and loweroverall probability of pathology, and smaller S indicates less stabilityand a greater overall probability of pathology.

$S = {1 - {\max\limits_{j}\left( P_{j} \right)}}$

In the second example, each separate pathology probability, Pj, istested against a probability threshold, tj, to decide whether the beaconexhibits the particular pathology with sufficient probability (seeequation below). The indicator function, I(X), takes value 1 if itsargument is true and 0 if its argument is false. Thus, if any pathologyprobability exceeds its threshold, then S=0. Otherwise, S=1.

$S = {1 - {\max\limits_{j}\left( {I\left( {P_{j} > t_{j}} \right)} \right)}}$

In the third example, each pathology has a corresponding weight, wj,related to its relative significance in determining overall stability(see equations below). For instance, in situations in which mobilebeacons have a higher probability of causing large client positioningerrors than Excess Range beacons, the probability of Mobility wouldreceive a higher weight than the probability Excess Range.

$S = {1 - {\sum\limits_{j}{w_{j}P_{j}}}}$${\sum\limits_{j}w_{j}} = 1$

Stability Applications and Extensions

Stability, as determined herein, is a valuable metric for evaluatingoverall beacon reliability both for client position estimation and forthe estimation and refinement of the positions of other beacons.Specifically, in certain implementations, stability is used in clientpositioning, database pruning, database expansion, and stabilitypropagation.

In client positioning, stability metrics are used to generate relativeweights to emphasize or deemphasize the contributions of individualbeacons to the overall position estimate. Similarly, stability is usedto emphasize or deemphasize the contributions of beacons used toposition other beacons. In either case, the overall stability of thebeacons used for location estimation are used to quantify the quality ofthe resulting estimate. That is, if the beacons used for positioning areextremely stable, then the estimated location should have acorrespondingly high reliability.

In database pruning, stability values are used to choose beacons forexclusion from the database. Stability values are also used to excludescans, relationships, or submissions that appear incorrect or spuriousbecause they contradict the attributes of an otherwise stable beacon.Database pruning, while not generally necessary, offers a means forsaving storage space and processing power in a large-scale positioningsystem.

Stability metrics are used in database expansion either as confidencefactors for stable bootstrapping or as indicators of the quality ofbeacons in a particular class. Database expansion through bootstrappinginvolves positioning beacons based on other beacons. Thus, if thebeacons used for positioning are unreliable, it can lead to “viraleffects” that degrade the overall quality of the system by propagatingunreliable beacon positions. Stability provides a means for ensuringthat beacons used for positioning other beacons are sufficientlyreliable.

Indicators of beacon class quality are important to database growth andmaintenance because they can indicate the need for measures such as thededicated scanning of an area or communication with a beacon-operatingentity in order to acquire new or corrected beacon data. For instance,if a given geographical area exhibits unacceptably low stability, thenit may be necessary to dispatch dedicated scanning devices to survey thearea and provide updated scans. Alternatively, if a set of beaconsbelonging to a particular entity has begun to exhibit unacceptably lowstability, then the entity could be contacted to request new submissionsof beacon information.

Finally, since the stability of a given beacon is often dependent on thestability of neighboring beacons, updating the stability values of abeacon or set of beacons allows the propagation of updated stabilityvalues to other beacons in the network.

Specifically, in some implementations, in a beacon graph, stability ispropagated from one beacon to another over the edges formed byrelationships. That is, the stability of a given beacon is reevaluatedusing the respective stabilities of the other beacons to which it isrelated and the attributes of the relationships connecting them.

For example, if the attributes of a relationship indicate that twobeacons are close together with extremely high probability, and one ofthe beacons exhibits an extremely high stability, then the relatedbeacon should also be considered highly stable. Conversely, if abeacon's estimated position and other attributes are derived from anensemble of relatively unstable neighbors, then the beacon itself shouldbe considered unstable.

Further, whenever a single beacon either enters the graph or has itsattributes updated, related beacons can update their own attributes inresponse, and the updates can propagate through the entire beacon graphat a rate determined by computational and other implementationconsiderations. One common technique for efficiently propagating updatesthrough a graph is to iteratively choose random sets of nodes to updateand continuing to iterate until a desired level of convergence isachieved.

Empirical Probabilities and Thresholding

In the examples provided above, certain probability functions and/orthreshold values can be derived from empirical data. The followingdescription provides techniques for determining these values as used incertain embodiments of the invention. The use of other techniques knownto one having ordinary skill in the art are also within the scope of theinvention.

In order to derive the empirical probability, P(a), of a given featureor event, a, it is necessary to first assemble a representative set oftraining data in which some samples (subset A) are known to exhibit theparticular feature and the remainder (subset B) are known to not exhibitthe feature. Thus, the probability of a given sample exhibiting thefeature is given by the size of subset A divided by the sum of the sizesof subsets A and B.

${P(a)} = \frac{A}{{A} + {B}}$

The probability expression can be further refined by finding theconditional probabilities of the feature given an observable, x.Specifically, given a sample that exhibits observable x, the probabilityof feature a can be found by dividing the number of samples in set Athat exhibit observable x, Ax, by the sum of the numbers of samples inboth A and B that exhibit x. In some circumstances, it may be morepractical to compute the equivalent expression in terms of theprobability of observable x given feature a, also given below.

${P\left( a \middle| x \right)} = {\frac{A_{x}}{{A_{x}} + {B_{x}}} = \frac{{P\left( x \middle| a \right)}{P(a)}}{P(x)}}$

Finally, to place a limit on the probability that a given sampleexhibits feature a, an empirical threshold on the value of observable xcan be derived. For example, if P(a|x) is non-decreasing with x, then wecan find threshold x′ such that P(a|x<x′)<p′. The value of x′ can bedetermined by testing the empirical values of P(a|x′) directly for arange of x′ or by deriving an analytical function to describe therelationship between P(a|x) and x.

The techniques and systems disclosed herein may be implemented as acomputer program product for use with a computer system or computerizedelectronic device. Such implementations may include a series of computerinstructions, or logic, fixed either on a tangible medium, such as acomputer readable medium (e.g., a diskette, CD-ROM, ROM, flash memory orother memory or fixed disk) or transmittable to a computer system or adevice, via a modem or other interface device, such as a communicationsadapter connected to a network over a medium.

The medium may be either a tangible medium (e.g., optical or analogcommunications lines) or a medium implemented with wireless techniques(e.g., Wi-Fi, cellular, microwave, infrared or other transmissiontechniques). The series of computer instructions embodies at least partof the functionality described herein with respect to the system. Thoseskilled in the art should appreciate that such computer instructions canbe written in a number of programming languages for use with manycomputer architectures or operating systems.

Furthermore, such instructions may be stored in any tangible memorydevice, such as semiconductor, magnetic, optical or other memorydevices, and may be transmitted using any communications technology,such as optical, infrared, microwave, or other transmissiontechnologies.

It is expected that such a computer program product may be distributedas a removable medium with accompanying printed or electronicdocumentation (e.g., shrink wrapped software), preloaded with a computersystem (e.g., on system ROM or fixed disk), or distributed from a serveror electronic bulletin board over the network (e.g., the Internet orWorld Wide Web). Of course, some embodiments of the invention may beimplemented as a combination of both software (e.g., a computer programproduct) and hardware. Still other embodiments of the invention areimplemented as entirely hardware, or entirely software (e.g., a computerprogram product).

Moreover, the techniques and systems disclosed herein can be used with avariety of mobile devices. For example, mobile telephones, smart phones,personal digital assistants, satellite positioning units (e.g., GPSdevices), and/or mobile computing devices capable of receiving thesignals discussed herein can be used in implementations of theinvention. The location estimate, expected error of the positionestimate, and/or the probability values can be displayed on the mobiledevice and/or transmitted to other devices and/or computer systems.Further, it will be appreciated that the scope of the present inventionis not limited to the above-described embodiments, but rather is definedby the appended claims; and that these claims will encompassmodifications of and improvements to what has been described.

1. A method of determining a measure of likelihood that a designatedwireless device exhibits at least one pathological characteristic, thepathological characteristic causing errors in position estimation tooccur when the designated wireless device is used as a landmark in aposition estimation system if left uncorrected, the method comprising:determining a set of one or more reference points associated with adesignated wireless device, one or more of the reference points being atleast one of (i) a geographic position at which signals from thedesignated wireless device were detected and (ii) another wirelessdevice from which signals were also detected by a receiver within aselected period of time during which the signals from the designatedwireless device were detected by the receiver; retrieving at least oneof (i) attributes of the reference points of the set, (ii) attributesassociated with the reception of signals from the designated wirelessdevice and the another wireless device within the selected period oftime, and (iii) statistical information about at least one of: atemporal distribution of times of detection of signals of referencepoints of the set, a spatial distribution of the reference points, and acardinality of the set or at least one subset of the reference points;and determining a measure of likelihood that the designated wirelessdevice exhibits at least one pathological characteristic based on atleast one of (i) attributes of the reference points, (ii) attributesassociated with the reception of signals, and (iii) the statisticalinformation.
 2. The method of claim 1, the measure of likelihood being abinary decision.
 3. The method of claim 1, the measure of likelihoodbeing a relative measure of a probability that the designated wirelessdevice exhibits the at least one pathological characteristic.
 4. Themethod of claim 1, further comprising determining a measure oflikelihood that a wireless device other than the designated wirelessdevice exhibits at least one pathological characteristic based on theother wireless device having a relationship with the designated wirelessdevice, the existence of the relationship being based on the designatedwireless device and the other wireless device being within signalreception range of a same position within a given period of time.
 5. Themethod of claim 1, the designated wireless device having a firstidentifier and the method further comprising determining a secondmeasure of likelihood that a second designated wireless device exhibitsat least one pathological characteristic based on a comparison of thefirst identifier and a second identifier associated with the seconddesignated wireless device.
 6. The method of claim 1, the at least onepathological characteristic including the designated wireless devicebeing a mobile device which lacks a fixed geographic position.
 7. Themethod of claim 1, the at least one pathological characteristicincluding the designated wireless device exhibiting signal transmissioncharacteristics that are atypical of a wireless device of a typematching the designated wireless device.
 8. The method of claim 1, theat least one pathological characteristic including that the locationinformation associated with the designated wireless device is not anaccurate present location of the wireless device
 9. The method of claim1, the at least one pathological characteristic including that anidentifier associated with the designated wireless devices is alsoassociated with at least one other wireless device.
 10. The method ofclaim 1, the designated wireless device having an identifier and themethod further comprising associating, in a reference database, theidentifier with an indication of the measure of likelihood.
 11. Themethod of claim 1, the designated wireless device being a WiFi-enabledaccess point.
 12. The method of claim 1, the designated wireless devicebeing a mobile telephone transceiver installation.
 13. The method ofclaim 1, for reference points of the set that are another wirelessdevice, the another wireless device of the set being at least one of aWiFi-enabled access point and a mobile telephone transceiverinstallation.
 14. The method of claim 1, further comprising: theattributes of the reference points of the set including locationinformation specifying estimated geographic locations of the referencepoints of the set; grouping each reference point of the set intoclusters of at least one reference point based on the locationinformation of the corresponding reference point; and for each cluster,determining attributes of the cluster based on at least one of (i)attributes of the reference points of the set, (ii) attributesassociated with the reception of signals from the designated wirelessdevice and the another wireless device within the selected period oftime; the determining the measure of likelihood being further based onthe attributes of at least one cluster.
 15. A system for determining ameasure of likelihood that a designated wireless device exhibits atleast one pathological characteristic, the pathological characteristiccausing errors in position estimation to occur when the designatedwireless device is used as a landmark in a position estimation system ifleft uncorrected, the system comprising: a computer readable mediaincluding instructions that when executed by a computer system cause thecomputer system to: determine a set of one or more reference pointsassociated with a designated wireless device, one or more of thereference points being at least one of (i) a geographic position atwhich signals from the designated wireless device were detected and (ii)another wireless device from which signals were also detected by areceiver within a selected period of time during which the signals fromthe designated wireless device were detected by the receiver; retrieveat least one of (i) attributes of the reference points of the set, (ii)attributes associated with the reception of signals from the designatedwireless device and the another wireless device within the selectedperiod of time, and (iii) statistical information about at least one of:a temporal distribution of times of detection of signals of referencepoints of the set, a spatial distribution of the reference points, and acardinality of the set or at least one subset of the reference points;and determine a measure of likelihood that the designated wirelessdevice exhibits at least one pathological characteristic based on atleast one of (i) attributes of the reference points, (ii) attributesassociated with the reception of signals, and (iii) the statisticalinformation.
 16. The system of claim 15, the measure of likelihood beinga binary decision.
 17. The system of claim 15, the measure of likelihoodbeing a relative measure of a probability that the designated wirelessdevice exhibits the at least one pathological characteristic.
 18. Thesystem of claim 15, the computer readable media further includinginstructions that when executed by a computer system cause the computersystem to determine a measure of likelihood that a wireless device otherthan the designated wireless device exhibits at least one pathologicalcharacteristic based on the other wireless device having a relationshipwith the designated wireless device, the existence of the relationshipbeing based on the designated wireless device and the other wirelessdevice being within signal reception range of a same position within agiven period of time.
 19. The system of claim 15, the designatedwireless device having a first identifier and the computer readablemedia further including instructions that when executed by a computersystem cause the computer system to determine a second measure oflikelihood that a second designated wireless device exhibits at leastone pathological characteristic based on a comparison of the firstidentifier and a second identifier associated with the second designatedwireless device.
 20. The system of claim 15, the at least onepathological characteristic including the designated wireless devicebeing a mobile device which lacks a fixed geographic position.
 21. Thesystem of claim 15, the at least one pathological characteristicincluding the designated wireless device exhibiting signal transmissioncharacteristics that are atypical of a wireless device of a typematching the designated wireless device.
 22. The system of claim 15, theat least one pathological characteristic including that the locationinformation associated with the designated wireless device is not anaccurate present location of the wireless device
 23. The system of claim15, the at least one pathological characteristic including that anidentifier associated with the designated wireless devices is alsoassociated with at least one other wireless device.
 24. The system ofclaim 15, the designated wireless device having an identifier and thecomputer readable media further including instructions that whenexecuted by a computer system cause the computer system to associate, ina reference database, the identifier with an indication of the measureof likelihood.
 25. The system of claim 15, the designated wirelessdevice being a WiFi-enabled access point.
 26. The system of claim 15,the designated wireless device being a mobile telephone transceiverinstallation.
 27. The system of claim 15, for reference points of theset that are another wireless device, the another wireless device of theset being at least one of a WiFi-enabled access point and a mobiletelephone transceiver installation.
 28. The system of claim 15, theattributes of the reference points of the set including locationinformation specifying estimated geographic locations of the referencepoints of the set, and the computer readable media further includinginstructions that when executed by a computer system cause the computersystem to: group each reference point of the set into clusters of atleast one reference point based on the location information of thecorresponding reference point; and for each cluster, determineattributes of the cluster based on at least one of (i) attributes of thereference points of the set, (ii) attributes associated with thereception of signals from the designated wireless device and the anotherwireless device within the selected period of time; the determining themeasure of likelihood being further based on the attributes of at leastone cluster.